Three-dimensional shape measuring apparatus, three-dimensional shape measuring method, program, and storage medium

ABSTRACT

A three-dimensional shape measuring apparatus includes: a single projector device that projects a first light pattern whose luminance changes at a first cycle and a second light pattern whose luminance changes at a second cycle that is longer than the first cycle on a measured object; an image capture device that acquire an image of the measured object on which the first or second light pattern is projected; and an image processing device that processes the image acquired by the image capture device. The image processing device includes a relative phase value calculation unit that calculates a relative phase value on each part of the measured object based on a luminance value of an image of the measured object on which the first light pattern is projected, an absolute phase value calculation unit that calculates an absolute phase value on each part of the measured object based on a luminance value and the relative phase value of an image of the measured object on which the second light pattern is projected, and a three-dimensional coordinate calculation unit that calculates three-dimensional coordinates at each part of the measured object based on the absolute phase value.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation application of U.S. patentapplication Ser. No. 16/980,978 filed on Sep. 15, 2020, which is aNational Stage Entry of international application PCT/JP2019/010414,filed on Mar. 13, 2019, which claims the benefit of priority fromJapanese Patent Application 2018-049546 filed on Mar. 16, 2018, thedisclosures of all of which are incorporated in their entirety byreference herein.

TECHNICAL FIELD

The present invention relates to a three-dimensional shape measuringapparatus, a three-dimensional shape measuring method, a program, and astorage medium.

BACKGROUND ART

Conventionally, various methods for measuring the three-dimensionalshape of an object have been proposed. For example, a method ofidentifying a three-dimensional shape by using a plurality of cameras tocapture a measured object and using a triangulation principle and amethod of identifying a three-dimensional shape by projecting apredetermined light pattern on a measured object and measuring aprojected light pattern are known. Patent Literatures 1 to 6 disclose ause of a sinusoidal pattern as a light pattern to be projected on ameasured object. By using a sinusoidal pattern for a light pattern to beprojected on a measured object, it is possible to identify the positionon the measured object based on a phase of a projected light (luminance)as a clue and accurately detect a three-dimensional shape of themeasured object. However, when a light pattern having periodicity suchas a sinusoidal pattern is used as a light pattern projected on ameasured object, since points shifted by N cycles may be a candidate dueto the nature thereof, it is difficult to uniquely define athree-dimensional shape.

Accordingly, Patent Literature 1 and the like solve the ambiguitydescribed above by adding one or more projectors that project lightpatterns or one or more cameras that capture a measured object touniquely define a three-dimensional shape. Further, in Patent Literature6 or the like, a three-dimensional shape is uniquely defined byprojecting two types of sinusoidal patterns having different cycles,respectively, to solve the ambiguity described above.

CITATION LIST Patent Literature

-   PTL 1: Japanese Patent Application Laid-Open No. 2001-012925-   PTL 2: Japanese Patent Application Laid-Open No. 2003-269928-   PTL 3: Japanese Patent Application Laid-Open No. 2006-214785-   PTL 4: Japanese Patent Application Laid-Open No. 2008-009807-   PTL 5: Japanese Patent Application Laid-Open No. 2009-115612-   PTL 6: Japanese Patent Application Laid-Open No. 2010-281178

SUMMARY OF INVENTION Technical Problem

The conventional method of resolving the ambiguity described above byprojecting two types of sinusoidal patterns having different cycles,respectively, has a problem of requiring a long time for measurementbecause the projection period of the light pattern is doubled.

The present invention intends to provide a three-dimensional shapemeasuring apparatus, a three-dimensional shape measuring method, aprogram, and a storage medium that may measure the three-dimensionalshape of an object at high accuracy and in a short time with a simplersystem.

Solution to Problem

According to one example aspect of the present invention, provided is athree-dimensional shape measuring apparatus including: a singleprojector device that projects a first light pattern whose luminancechanges at a first cycle and a second light pattern whose luminancechanges at a second cycle that is longer than the first cycle on ameasured object; an image capture device that acquires an image of themeasured object on which the first light pattern or the second lightpattern is projected; and an image processing device that processes theimage acquired by the image capture device, and the image processingdevice includes a relative phase value calculation unit that, based on aluminance value at each of pixels of an image of the measured object onwhich the first light pattern is projected, calculates a relative phasevalue on each part of the measured object corresponding to each of thepixels, an absolute phase value calculation unit that, based on aluminance value and the relative phase value at each of pixels of animage of the measured object on which the second light pattern isprojected, calculates an absolute phase value on each the part of themeasured object corresponding to each of the pixels, and athree-dimensional coordinate calculation unit that, based on theabsolute phase value, calculates three-dimensional coordinates at eachthe part of the measured object corresponding to each of the pixels.

Further, according to another example aspect of the present invention,provided is a three-dimensional shape measuring method including thesteps of: projecting a first light pattern whose luminance changes at afirst cycle on a measured object and acquiring an image of the measuredobject on which the first light pattern is projected; projecting asecond light pattern whose luminance changes at a second cycle that islonger than the first cycle on the measured object by the same projectordevice as a projector device used for projection of the first lightpattern and acquiring an image of the measured object on which thesecond light pattern is projected; based on a luminance value at each ofpixels of an image of the measured object on which the first lightpattern is projected, calculating a relative phase value on each part ofthe measured object corresponding to each of the pixels; based on aluminance value and the relative phase value at each of pixels of animage of the measured object on which the second light pattern isprojected, calculating an absolute phase value on each the part of themeasured object corresponding to each of the pixels; and based on theabsolute phase value, calculating three-dimensional coordinates at eachthe part of the measured object corresponding to each of the pixels.

Further, according to yet another example aspect of the presentinvention, provided is a program that causes a computer to function as:a unit that controls a single projector device to project a first lightpattern whose luminance changes at a first cycle or a second lightpattern whose luminance changes at a second cycle that is longer thanthe first cycle on a measured object; a unit that acquires an image ofthe measured object on which the first light pattern is projected and animage of the measured object on which the second light pattern isprojected; a unit that, based on a luminance value at each of pixels ofan image of the measured object on which the first light pattern isprojected, calculates a relative phase value on each part of themeasured object corresponding to each of the pixels; a unit that, basedon a luminance value and the relative phase value at each of pixels ofan image of the measured object on which the second light pattern isprojected, calculates an absolute phase value on each the part of themeasured object corresponding to each of the pixels; and a unit that,based on the absolute phase value, calculates three-dimensionalcoordinates at each the part of the measured object corresponding toeach of the pixels.

Advantageous Effects of Invention

According to the present invention, it is possible to measure athree-dimensional shape of a measured object at high accuracy and in ashort time with a simpler system.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic diagram illustrating a configuration example of athree-dimensional shape measuring apparatus according to a first exampleembodiment of the present invention.

FIG. 2A is a diagram illustrating an example of a short cycle lightpattern used in a three-dimensional shape measuring method according tothe first example embodiment of the present invention.

FIG. 2B is a diagram illustrating an example of a short cycle lightpattern used in the three-dimensional shape measuring method accordingto the first example embodiment of the present invention.

FIG. 2C is a diagram illustrating an example of a short cycle lightpattern used in the three-dimensional shape measuring method accordingto the first example embodiment of the present invention.

FIG. 2D is a diagram illustrating an example of a short cycle lightpattern used in the three-dimensional shape measuring method accordingto the first example embodiment of the present invention.

FIG. 3A is a diagram illustrating an example of a long cycle lightpattern used in the three-dimensional shape measuring method accordingto the first example embodiment of the present invention.

FIG. 3B is a diagram illustrating an example of a long cycle lightpattern used in the three-dimensional shape measuring method accordingto the first example embodiment of the present invention.

FIG. 4 is a graph illustrating a method of calculating an absolute phasevalue in the three-dimensional shape measuring method according to thefirst example embodiment of the present invention.

FIG. 5 is a flowchart illustrating the three-dimensional shape measuringmethod according to the first example embodiment of the presentinvention.

FIG. 6A is a diagram illustrating an example of a long cycle lightpattern used in a three-dimensional shape measuring method according toa second example embodiment of the present invention.

FIG. 6B is a diagram illustrating an example of a long cycle lightpattern used in a three-dimensional shape measuring method according tothe second example embodiment of the present invention.

FIG. 7 is a graph illustrating a method of capturing an absolute phasevalue in the three-dimensional shape measuring method according to thesecond example embodiment of the present invention.

FIG. 8 is a flowchart illustrating the three-dimensional shape measuringmethod according to the second example embodiment of the presentinvention.

FIG. 9A is a graph illustrating a result of inspection of anadvantageous effect of the present invention inspected through acomputer simulation.

FIG. 9B is a graph illustrating a result of inspection of anadvantageous effect of the present invention inspected through acomputer simulation.

FIG. 10 is a schematic diagram illustrating a configuration example of athree-dimensional shape measuring apparatus according to a third exampleembodiment of the present invention.

FIG. 11A is a diagram illustrating an example of a light patternaccording to a modified example of the example embodiment of the presentinvention.

FIG. 11B is a diagram illustrating an example of a light patternaccording to a modified example of the example embodiment of the presentinvention.

DESCRIPTION OF EMBODIMENTS First Example Embodiment

A three-dimensional shape measuring apparatus and a three-dimensionalshape measuring method according to a first example embodiment of thepresent invention will be described with reference to FIG. 1 to FIG. 5.

FIG. 1 is a schematic diagram illustrating a configuration example ofthe three-dimensional shape measuring apparatus according to the presentexample embodiment. FIG. 2A to FIG. 2D are diagrams each illustrating anexample of a short cycle light pattern used in the three-dimensionalshape measuring method according to the present example embodiment. FIG.3A and FIG. 3B are diagrams each illustrating an example of a long cyclelight pattern used in the three-dimensional shape measuring methodaccording to the present example embodiment. FIG. 4 is a graphillustrating a calculation method of an absolute phase value in thethree-dimensional shape measuring method according to the presentexample embodiment. FIG. 5 is a flowchart illustrating thethree-dimensional shape measuring method according to the presentexample embodiment.

As illustrated in FIG. 1, a three-dimensional shape measuring apparatus100 according to the present example embodiment includes a projectordevice 20, an image capture device 30, and an image processing device40. The image processing device 40 includes a projection pattern controlunit 42, an image acquisition unit 44, a relative phase valuecalculation unit 46, an absolute phase value calculation unit 48, athree-dimensional coordinate calculation unit 50, and a control unit 52.

The projector device 20 is a device that projects a predetermined lightpattern on a measured object 10 whose three-dimensional shape is to bemeasured. The projector device 20 is not particularly limited and maybe, for example, a digital light processing (DLP) projector, a liquidcrystal projector, or the like. A DLP projector or a liquid crystalprojector can project any light patterns at a high rate and ispreferable for reducing time required for a three-dimensional shapemeasurement. Reduction in the measuring time is preferable for measuringa three-dimensional shape of an object that is moving (moving object)such as when face authentication of a person is performed, inparticular. Note that details of the light pattern projected on themeasured object 10 will be described later. The measured object 10 maybe a face, a head, a finger, a fingerprint, a ridge of a fingerprint, orother parts including a part of the body of a person but is not limitedthereto.

The image capture device 30 is a device that captures an image of themeasured object 10 on which a light pattern emitted from the projectordevice 20 is projected. The image capture device 30 includes asolid-state image pickup device such as a charge coupled device (CCD)image sensor or a complementary metal-oxide-semiconductor (CMOS) imagesensor. Further, the image capture device 30 includes an optical systemthat captures an image of a subject on a capturing plane of asolid-state image pickup device, a signal processing circuit thatperforms signal processing on the output of the solid-state image pickupdevice to obtain a luminance value on a pixel basis, and the like.

The image processing device 40 may be formed of a general purposeinformation processing device (computer) having a central processingunit (CPU), a memory device, a display, a storage device such as a harddisk, various interfaces for input/output, or the like. Further, theimage processing device 40 has a program that causes the informationprocessing device to perform a three-dimensional shape measuring methoddescribed later, and when the CPU executes the program, the function ofeach unit of the image processing device 40 can be implemented.

The projection pattern control unit 42 has a function of generating alight pattern to be projected on the measured object 10 and storing thelight pattern in the storage device in advance. Further, the projectionpattern control unit 42 has a function of transmitting data of a lightpattern stored in the storage device to the projector device 20 via ageneral purpose display interface such as a digital visual interface(DVI). Further, the projection pattern control unit 42 has a function ofcontrolling the operation of the projector device 20 (turning on/off,dimming adjustment, or the like of a light source) via a general purposecommunication interface such as RS232 or universal serial bus (USB).

The image acquisition unit 44 has a function of acquiring image datacaptured by the image capture device 30 and storing the image data in amemory device. Further, the image acquisition unit 44 has a function ofcontrolling of the operation of the image capture device 30 (a timing ofcapturing or the like) via a general purpose communication interfacesuch as RS232 or USB. Further, the image acquisition unit 44 has afunction of instructing the projection pattern control unit 42 to causethe projector device 20 to project a light pattern.

The relative phase value calculation unit 46 has a function ofcalculating a phase value in accordance with a luminance value (phasevalues θ and φ described later) on a pixel basis based on an imagecaptured by the image capture device 30. Note that details of a methodof calculating a phase value will be described later.

The absolute phase value calculation unit 48 has a function ofcalculating an absolute phase value (an absolute phase value Θ describedlater) on a pixel basis based on a phase value calculated by therelative phase value calculation unit 46. Note that details of a methodof calculating a phase value will be described later.

The three-dimensional coordinate calculation unit 50 has a function offinding three-dimensional coordinates of a projection point P (X, Y, Z)on the measured object 10 corresponding to a point p(x, y) on an imageby using calculation based on an absolute phase value of each pixelcalculated by the absolute phase value calculation unit 48.

The control unit 52 has a function of generally controlling the aboveunits of the image processing device 40.

Next, before the three-dimensional shape measuring method according tothe present example embodiment is specifically described, a basicprinciple of a three-dimensional shape measuring method using asinusoidal grating phase shift method will be described.

The sinusoidal grating phase shift method is a method of projecting asinusoidal grating light pattern as illustrated in FIG. 2A to FIG. 2D(hereafter, referred to as a sinusoidal pattern) on the measured object10 while gradually shifting the phase and identifying thethree-dimensional shape based on captured images of the measured object10 on which the light pattern is projected.

FIG. 2A to FIG. 2D illustrate four sinusoidal patterns whose phases areeach shifted by ¼ wavelengths as an example. Each drawing of FIG. 2A toFIG. 2D represents the luminance in a projection region of the lightpattern in grayscale. In the sinusoidal pattern illustrated in eachdrawing of FIG. 2A to FIG. 2D, the luminance changes sinusoidally in thevertical direction of the drawing. FIG. 2A illustrates a light patternprojected at time t=0, FIG. 2B illustrates a light pattern projected attime t=π/2, FIG. 2C illustrates a light pattern projected at time t=π,and FIG. 2D illustrates a light pattern projected at time t=3π/2.

Note that a variable t is here denoted as “time”, description isprovided for a case of modulating the amplitude of the luminance. Inimplementation, a vertical displacement of the phase when the lightpattern of FIG. 2A is a reference is illustrated, and the projectiontiming of a light pattern can be changed regardless of the actual time.The same applies to FIG. 3A and FIG. 3B described later.

When the measured object 10 on which a sinusoidal pattern is projectedby the image capture device 30, a luminance value I(x, y, t) at time tat (x, y) coordinates of an obtained image is expressed as Equation (1)below, where the amplitude of a sine wave is denoted as A, the phasevalue is denoted as θ, and the bias (the center value of a sine wave) isdenoted as B.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 1} \right\rbrack & \; \\{{I\left( {x,y,t} \right)} = {{A \cdot {\cos\left( {t + \theta} \right)}} + B}} & (1)\end{matrix}$

Since the light pattern projected by the projector device 20 differs inthe phase value θ for each angle viewed from the projector device 20, ifthe phase value θ at coordinates (x, y) can be calculated, thethree-dimensional position corresponding to the coordinates (x, y) canbe defined.

Since Equation (1) has three unknowns of the amplitude A, the phasevalue θ, and the bias B, with at least three light pattern projectionimages, the phase value θ can be calculated. When four or more lightpattern projection images are captured, the phase value θ can becalculated more accurately by using a least-squares method or the like.

Herein, if images at time t=0, time t=π/2, time t=π, and time t=3π/2 areacquired, the luminance value I(x, y, t) at the coordinates (x, y) ateach time is expressed as Equation (2) to Equation (5) below.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 2} \right\rbrack & \; \\{{I\left( {x,y,0} \right)} = {{A \cdot {\cos(\theta)}} + B}} & (2) \\{{I\left( {x,y,\frac{\pi}{2}} \right)} = {{A \cdot {\cos\left( {\frac{\pi}{2} + \theta} \right)}} + B}} & (3) \\{{I\left( {x,y,\ \pi} \right)} = {{A \cdot {\cos\left( {\pi + \theta} \right)}} + B}} & (4) \\{{I\left( {x,y,\frac{3\pi}{2}} \right)} = {{A \cdot {\cos\left( {\frac{3\pi}{2} + \theta} \right)}} + B}} & (5)\end{matrix}$

Equation (6) to Equation (8) below are obtained by finding the amplitudeA, the phase value θ, and the bias B by using a least-squares methodfrom Equation (2) to Equation (5).

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 3} \right\rbrack & \; \\{A = \frac{\sqrt{\left( {{I\left( {x,y,0} \right)} - {I\left( {x,y,\pi} \right)}} \right)^{2} + \left( {{I\left( {x,y,\frac{\pi}{2}} \right)} - {I\left( {x,y,\frac{3\pi}{2}} \right)}} \right)^{2}}}{2}} & (6) \\{B = \frac{{I\left( {x,y,0} \right)} + {I\left( {x,y,\frac{\pi}{2}} \right)} + {I\left( {x,y,\pi} \right)} + {I\left( {x,y,\frac{3\pi}{2}} \right)}}{4}} & (7) \\{\theta = {{\tan\;}^{- 1}\frac{{- {I\left( {x,y,\frac{\pi}{2}} \right)}} + {I\left( {x,y,\frac{3\pi}{2}} \right)}}{{I\left( {x,y,0} \right)} - {I\left( {x,y,\pi} \right)}}}} & (8)\end{matrix}$

In general, in the sinusoidal grating phase shift method, it is knownfrom experience that resolution around 1/200 of the depth correspondingto one cycle is obtained, and accuracy of around 100 μm to 200 μm isactually obtained as an example.

Since the sine wave is a repeating function, however, the obtained phasevalue θ ranges from −π to π (−π≤θ≤π). Therefore, points indicating thesame phase value θ may be present, the number of which is the same asthe number of cycles included in a projection region of a light pattern,and coordinates cannot be uniquely defined from the obtained phase valueθ. Although the use of a sinusoidal pattern where the whole screen of acaptured image corresponds to one cycle can solve this uncertainty,there is a tradeoff relationship where depth measuring accuracydeteriorates accordingly. Thus, conventionally, a so-called multi-eyesinusoidal grating phase shift method in which images acquired by twoimage capture devices are used to solve the uncertainty is used.However, the multi-eye sinusoidal grating phase shift method uses aplurality of projectors or a plurality of image capture devices and thushas a problem of system configuration or control being more complex orthe like.

In the three-dimensional shape measuring method according to the presentexample embodiment, two types of light patterns are used as the lightpattern to be projected on the measured object 10. These two types oflight patterns are periodic light patterns whose numbers of repeatingcycles are different from each other, which are distinguished here as ashort cycle light pattern and a long cycle light pattern. That is, thethree-dimensional shape measuring method according to the presentexample embodiment is to perform measurement of a phase value using along cycle light pattern in addition to measurement of a phase valueusing a short cycle light pattern similar to the measurement by usingthe sinusoidal phase shift method described above.

In the present example embodiment, a sinusoidal pattern is used as eachof the short cycle light pattern and the long cycle light pattern. Theshort cycle light pattern is a light pattern that determines theresolution of measurement. The greater the number of cycles of the shortcycle light pattern included in the whole screen of an image captured bythe image capture device 30 is, the higher the resolution of measurementis. Thus, it is desirable that the short cycle light pattern be a lightpattern including multiple cycles of patterns in a whole screen of acaptured image, as illustrated in FIG. 2A to FIG. 2D, for example. Thenumber of cycles of a short cycle light pattern included in the wholescreen can be suitably set in accordance with resolution required formeasurement. The long cycle light pattern is a light pattern used foridentifying the absolute phase value Θ and desirably a light patternincluding one or less cycle of pattern in the whole screen of an imagecaptured by the image capture device 30.

As illustrated in FIG. 3A and FIG. 3B, for example, a light patternincluding one cycle of pattern in the whole screen of a captured imagecan be applied to the long cycle light pattern. Each drawing of FIG. 3Aand FIG. 3B represents the luminance in the projection region of a lightpattern in grayscale in the same manner as FIG. 2A to FIG. 2D. Also inthe sinusoidal pattern illustrated in FIG. 3A and FIG. 3B, the luminancechanges sinusoidally in the vertical direction of the drawing. FIG. 3Aillustrates a light pattern projected at time t=0, and FIG. 3Billustrates a light pattern projected at time t=π/2.

When the measured object 10 on which the short cycle light pattern isprojected is captured by the image capture device 30, the luminancevalue I(x, y, t) at time t at (x, y) coordinates of an obtained image isexpressed as Equation (1) described above, where the amplitude isdenoted as A, the phase value is denoted as θ, and the bias is denotedas B.

Herein, if images at time t=0, time t=π/2, time t=π, and time t=3π/2 areacquired, the luminance value I(x, y, t) at the coordinates (x, y) isexpressed as Equation (2) to Equation (5) described above. Further,Equation (6) to Equation (8) described above are obtained by finding theamplitude A, the phase value θ, and the bias B by using a least-squaresmethod from Equation (2) to Equation (5).

On the other hand, when the measured object 10 on which the long cyclelight pattern is projected is captured by the image capture device 30,the luminance value J(x, y, t) at time t at (x, y) coordinates of anobtained image is expressed as Equation (9) below, where the amplitudeis denoted as A′, the phase value is denoted as φ, and the bias isdenoted as B′.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 4} \right\rbrack & \; \\{{J\left( {x,y,t} \right)} = {{A^{\prime} \cdot {\cos\left( {t + \varphi} \right)}} + B^{\prime}}} & (9)\end{matrix}$

Herein, if images at time t=0, time t=π/2, time t=π, and time t=3π/2 areacquired, the luminance value J(x, y, t) at the coordinates (x, y) isexpressed as Equation (10) to Equation (13) below.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 5} \right\rbrack & \; \\{{J\left( {x,y,\ 0} \right)} = {{A^{\prime} \cdot {\cos(\varphi)}} + B^{\prime}}} & (10) \\{{J\left( {x,y,\frac{\pi}{2}} \right)} = {{A^{\prime} \cdot {\cos\left( {\frac{\pi}{2} + \varphi} \right)}} + B^{\prime}}} & (11) \\{{J\left( {x,y,\ \pi} \right)} = {{A^{\prime} \cdot {\cos\left( {\pi + \varphi} \right)}} + B^{\prime}}} & (12) \\{{J\left( {x,y,\frac{3\pi}{2}} \right)} = {{A^{\prime} \cdot {\cos\left( {\frac{3\pi}{2} + \varphi} \right)}} + B^{\prime}}} & (13)\end{matrix}$

Equation (14) to Equation (16) below are obtained by finding theamplitude A′, the phase value φ, and the bias B′ by using aleast-squares method from Equation (10) to Equation (13).

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 6} \right\rbrack & \; \\{A^{\prime} = \frac{\sqrt{\left( {{J\left( {x,y,0} \right)} - {J\left( {x,y,\pi} \right)}} \right)^{2} + \left( {{J\left( {x,y,\frac{\pi}{2}} \right)} - {J\left( {x,y,\frac{3\pi}{2}} \right)}} \right)^{2}}}{2}} & (14) \\{B^{\prime} = \frac{{J\left( {x,y,0} \right)} + {J\left( {x,y,\frac{\pi}{2}} \right)} + {J\left( {x,y,\pi} \right)} + {J\left( {x,y,\frac{3\pi}{2}} \right)}}{4}} & (15) \\{\varphi = {\tan^{- 1}\frac{{- {J\left( {x,y,\frac{\pi}{2}} \right)}} + {J\left( {x,y,\frac{3\pi}{2}} \right)}}{{J\left( {x,y,0} \right)} - {J\left( {x,y,\pi} \right)}}}} & (16)\end{matrix}$

In general, in the sinusoidal grating phase shift method, it is knownfrom experience that resolution around 1/200 of the depth correspondingto a width of one cycle is obtained. In the case of four-time capturingas described above, when the design strategy is directed to the abilityof determining the absolute phase at a high certainty degree is a designstrategy, the absolute phase value can be determined from the relativephase value found from around ±3a, that is, (6/200)⁻¹, namely, around 33repeating sine waves.

Herein, when a projector that can project any light pattern, such as aliquid crystal projector or a DLP projector is used as the projectordevice 20, it is possible to switch a short cycle light pattern and along cycle light pattern at a high rate for projection by using a singleprojector device 20. Then, when the short cycle light pattern and thelong cycle light pattern generated based on the light emitted from thesame light source from the single projector device 20 are projected, itcan be assumed that the basic physical characteristics of the projectordevice 20 at projection of these light patterns are the same. That is,when projection on the measured object 10 of the short cycle lightpattern and the long cycle light pattern is performed by using the sameprojector device 20, it is considered that Equation (17) below is met.

$\begin{matrix}{{A = A^{\prime}},{B = B^{\prime}}} & (17)\end{matrix}$

Therefore, Equation (9) can be rewritten as Equation (18) and Equation(19) below, and the number of unknowns is not three, namely, theamplitude A′, the phase value φ, and the bias B′ but is one, namely,only the phase value φ. Accordingly, with at least only one time ofcapturing the measured object 10 on which the long cycle light patternis projected, the phase value φ can be found.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 7} \right\rbrack & \; \\{{J\left( {x,y,0} \right)} = {{A \cdot {\cos(\varphi)}} + B}} & (18) \\{\varphi = {\cos^{- 1}\frac{{J\left( {x,y,0} \right)} - B}{A}}} & (19)\end{matrix}$

In view of the above, in the present example embodiment, the short cyclelight pattern and the long cycle light pattern emitted from the singleprojector device 20 are projected on the measured object 10, which makesit possible to calculate the phase value φ while reducing the number oftimes of projection on the measured object 10. Accordingly, thethree-dimensional shape of the measured object 10 can be measured in ashort time.

By performing two times of capturing the measured object 10 on which thelong cycle light pattern is projected and finding the phase value φ by aleast-squares method, it is possible to improve the measurement accuracyof the phase value cp. Also in such a case, since the number ofcapturing times is reduced compared to the conventional method thatrequires at least three times of capturing, the measurement time of thethree-dimensional shape of the measured object 10 can be reduced.

When two times of capturing are performed with a shift of a half cycle,that is, if images at time t=0 and time t=n are obtained, respectively,a luminance value J(x, y, t) at the coordinates (x, y) is expressed byEquation (20) and Equation (21) below.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 8} \right\rbrack & \; \\{{J\left( {x,y,0} \right)} = {{A^{\prime} \cdot {\cos(\varphi)}} + B^{\prime}}} & (20) \\{{J\left( {x,y,\pi} \right)} = {{A^{\prime} \cdot {\cos\left( {\pi + \varphi} \right)}} + B^{\prime}}} & (21)\end{matrix}$

By finding the phase value φ from Equation (20) and Equation (21) by aleast-squares method, Equation (22) below is obtained.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 9} \right\rbrack & \; \\{\varphi = {\cos^{- 1}\frac{{J\left( {x,y,0} \right)} - {J\left( {x,y,\pi} \right)}}{2A}}} & (22)\end{matrix}$

It should be noted, however, that the uncertainty of the phase value φobtained by Equation (19) and Equation (22) is not a fraction dividedfrom 2η by an integer but a fraction divided from π by an integer (therange of the obtained phase value φ is 0≤φ≤π). That is, it is requiredto project sine waves so that the whole image corresponds not to onecycle but to a half cycle. This means that the relative error of thephase value φ becomes two times.

In order that the phase value φ can be used in a range of 2π, Equation(19) or Equation (22) used for finding the phase value φ needs to berewritten to be not a cosine function but an inverse function of atangent function. When capturing with a shift of ¼ cycles, that is,two-time capturing at time t=0 and time t=π/2 is assumed, the luminancevalue J(x, y, t) at the coordinates (x, y) is expressed by Equation (23)and Equation (24) below.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 10} \right\rbrack & \; \\{{J\left( {x,y,0} \right)} = {{A^{\prime} \cdot {\cos(\varphi)}} + B^{\prime}}} & (23) \\{{J\left( {x,\ y,\frac{\pi}{2}} \right)} = {{A^{\prime} \cdot {\cos\left( {\frac{\pi}{2} + \varphi} \right)}} + B^{\prime}}} & (24)\end{matrix}$

By finding the phase value φ from Equation (23) and Equation (24) by aleast-squares method, Equation (25) below is obtained. The uncertaintyof the phase value φ obtained therefrom is a fraction divided from 2π byan integer (the range of the obtained phase value φ is −π≤φ≤π).

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 11} \right\rbrack & \; \\{\varphi = {\tan^{- 1}\frac{{- {J\left( {x,y,\frac{\pi}{2}} \right)}} + B}{{J\left( {x,y,0} \right)} - B}}} & (25)\end{matrix}$

Note that the phase value of the long cycle light pattern to beprojected changes gradually in the whole image and thus is expected tohave a good compatibility with noise removal such as smoothing or amedian filter. With design to have the same absolute phase determinationaccuracy as that when four-time capturing is performed, it is possibleto cancel the disadvantage caused by a reduction of the number ofcapturing times.

The phase value θ becomes a value for every one cycle of the short cyclelight pattern, that is, a value from −π to π. Therefore, to find anabsolute phase value in a light pattern projected for multiple cycles, aprocess of estimating where a pattern of the order n (a value indicatingthe n-th cycle counted from one end to the other end) is located on eachcaptured image is required. In the present example embodiment, the phasevalue φ is used for the estimation of the order n of a short cycle lightpattern.

FIG. 4 is a graph illustrating an example of the relationship betweenthe relative phase value (the phase value θ and the phase value φ) andthe order n of a short cycle light pattern when the short cycle lightpattern within a screen includes 10 cycles and the long cycle lightpattern has one cycle.

As illustrated in FIG. 4, the order n of the short cycle light patterncan be determined uniquely in accordance with a value of the phase valuecp. For example, the order n of a short cycle light pattern is two in arange of the phase value φ being π/5 to 2π/5, and the order n of a shortcycle light pattern is seven in a range of the phase value φ being −4π/5to −3π/5.

With the order n of the short cycle light pattern being found, theabsolute phase value Θ (=θ+2π(n−1)) can be calculated from the phasevalue θ and the order n. The line obtained by connecting points havingthe same absolute phase value Θ on a captured image (equal phase line)represents a shape of a cross section of the measured object 10 takenalong a certain plane similar to a sectional line by a light-sectionmethod. Based on this absolute phase value Θ, the three-dimensionalshape of the measured object 10 (height information at each point of animage) can be measured by a triangulation principle. That is, it ispossible to identify the three-dimensional shape of the measured object10 by finding the absolute coordinate value of a projection point on themeasured object in a three-dimensional space corresponding to each pixelon an image by a triangulation principle based on the absolute phasevalue Θ and the optical arrangement of the projector device 20 and theimage capture device 30.

Since the three-dimensional shape measuring method of the presentexample embodiment uses two light patterns, namely, the short cyclelight pattern and the long cycle light pattern, the capturing time islonger than in the case where one type of the light pattern is used.However, with a use of a DLP projector, a liquid crystal projector, orthe like for projection of a light pattern, the capturing time can besufficiently reduced for the absolute time. Further, according to thethree-dimensional shape measuring method of the present exampleembodiment, it is expected to solve the problems of long calculationtime of three-dimensional coordinates and inability of measurement ifthe positional relationship between the camera and the projector on theleft and right becomes wrong, which are disadvantages of the multi-eyesinusoidal grating phase shift method described above.

Further, in the conventional method, when two light patterns, namely,the short cycle light pattern and the long cycle light pattern are used,projection and capturing of the light patterns are required to beperformed for at least three times, respectively, that is, six times intotal. In contrast, in the three-dimensional shape measuring methodaccording to the present example embodiment, projection and capturing ofthe light patterns need to be performed only for at least three times byusing the short cycle light pattern and one time by using the long cyclelight pattern, that is, four times in total. Therefore, according to thethree-dimensional shape measuring method of the present exampleembodiment, the number of images to be projected can be reduced, and thecapturing time can be shortened.

Note that, although measurement using a short cycle light pattern isperformed earlier and the amplitude A and the bias B are calculated inthe above description, measurement using a long cycle light pattern maybe performed earlier and the amplitude A′ and the bias B′ may becalculated.

The three-dimensional shape measuring method according to the presentexample embodiment can be performed in accordance with step S101 to stepS116 illustrated in FIG. 5, for example. Note that description isprovided here for a case of calculating the amplitude A, the bias B, andthe phase value θ from an image of the measured object 10 on which thefirst light pattern, which is the short cycle light pattern, isprojected and calculating the phase value φ from an image of themeasured object 10 on which the second light pattern, which is the longcycle light pattern, is projected, as an example. However, the amplitudeA′, the bias B′, and the phase value φ may be calculated from an imageof the measured object 10 on which the first light pattern, which is thelong cycle light pattern, is projected and calculate the phase value θfrom an image of the measured object 10 on which the second lightpattern, which is the short cycle light pattern, is projected.Projection of the second light pattern may be performed earlier thanprojection of the first light pattern.

First, in step S101, the number of projection times L for the firstlight pattern is set. Measurement using the first light pattern is fordetermining the amplitude A, the bias B, and the phase value θ, and thenumber of projection times L is greater than or equal to three. Toperform more accurate measurement by using a least-squares method or thelike, it is desirable that the number of projection times L be greaterthan or equal to four. As an example here, the number of projectiontimes L for the first light pattern is four.

Next, in step S102, under the control of the projection pattern controlunit 42, the projector device 20 projects the first light pattern on themeasured object 10 to be measured. As an example here, the first lightpattern, which is the short cycle light pattern, is projected on themeasured object 10. The pattern of FIG. 2A is applicable as the shortcycle light pattern, for example.

Next, in step S103, under the control of the image acquisition unit 44,the image capture device 30 captures an image of the measured object 10on which the first light pattern is projected.

Next, in step S104, the number of projection times L for the first lightpattern is decremented by one. The number of projection times Lrepresents the number of remaining projection times of the first lightpattern.

Next, in step S105, it is determined whether or not the number ofprojection times L of the first light pattern is zero, that is, whetheror not the projection and capturing of the first light pattern have beenperformed for the number of projection times L set in step S101.

As a result of the determination of step S105, if the number ofprojection times L is not zero (step S105, “No”), in step S106, thephase of the first light pattern projected on the measured object 10 isshifted, and the process returns to step S102. For example, when thenumber of projection times L is four, the projection pattern controlunit 42 sequentially prepares data of the short cycle light patternshaving phases shifted stepwise by ¼ wavelengths (see FIG. 2B, FIG. 2C,and FIG. 2D) and transmits the data to the projector device 20.

As a result of the determination in step S105, if the number ofprojection times L is zero (step S105, “Yes”), the process proceeds tostep S107.

Next, in step S107, the relative phase value calculation unit 46calculates the amplitude A, the bias B, and the phase value θ,respectively, based on the luminance value I of each pixel of the Limages captured in step S103. The amplitude A, the bias B, and the phasevalue θ can be calculated based on Equation (6) to Equation (8), forexample.

Next, in step S108, the number of projection times M for the secondlight pattern is set. Measurement using the second light pattern is fordetermining the phase value φ, and the number of projection times M isgreater than or equal to one. To perform more accurate measurement byusing a least-squares method or the like, it is desirable that thenumber of projection times M be greater than or equal to two. As anexample here, the number of projection times M for the second lightpattern is two.

Next, in step S109, under the control of the projection pattern controlunit 42, the projector device 20 projects the second light pattern onthe measured object 10 to be measured. As an example here, the secondlight pattern that is the long cycle light pattern is projected on themeasured object 10. The pattern of FIG. 3A is applicable as the longcycle light pattern, for example.

Next, in step S110, under the control of the image acquisition unit 44,the image capture device 30 captures an image of the measured object 10on which the second light pattern is projected.

Next, in step S111, the number of projection times M for the secondlight pattern is decremented by one. The number of projection times Mrepresents the number of remaining projection times of the second lightpattern.

Next, in step S112, it is determined whether or not the number ofprojection times M of the second light pattern is zero, that is, whetheror not the projection and capturing of the second light pattern havebeen performed for the number of projection times M set in step S108.

As a result of the determination of step S112, if the number ofprojection times M is not zero (step S112, “No”), in step S113, thephase of the second light pattern projected on the measured object 10 isshifted, and the process returns to step S109. For example, when thenumber of projection times M is two, the projection pattern control unit42 prepares data of the long cycle light patterns having phases shiftedstepwise by ¼ wavelengths (see FIG. 3B) and transmits the data to theprojector device 20.

As a result of the determination in step S112, if the number ofprojection times M is zero (step S112, “Yes”), the process proceeds tostep S114.

Next, in step S114, the relative phase value calculation unit 46calculates the phase value φ based on the amplitude A and the bias Bcalculated in step S107 and the luminance value J of each pixel of the Mimages captured in step S110, respectively. The phase value φ can becalculated based on Equation (25), for example.

Next, in step S115, the absolute phase value calculation unit 48calculates the absolute phase value Θ based on the phase value θcalculated in step S107 and the phase value φ calculated in step S114.

Next, in step S116, based on the absolute phase value Θ calculated instep S115, the three-dimensional coordinate calculation unit 50calculates the absolute coordinate values of the projection point on themeasured object 10 in the three-dimensional space corresponding to eachpixel of the captured image. Accordingly, the three-dimensional shape ofthe measured object 10 can be identified.

As described above, the three-dimensional measuring method according tothe present example embodiment can measure the three-dimensional shapeof a measured object by performing projection and capturing of lightpatterns for at least three times using the short cycle light patternand once using the long cycle light pattern, namely, four times intotal. Therefore, the three-dimensional measuring method according tothe present example embodiment can reduce the number of projected imagesand thus can shorten the measuring time compared to the conventionalmethod that requires projection and capturing of light patterns for atleast three times using the short cycle light pattern and three timesusing the long cycle light pattern, namely, six times in total. It isrequired to complete the measurement in a short time when athree-dimensional shape of a moving measured target is measured, inparticular, such as when face authentication of a person is performed,for example. The three-dimensional measuring method according to thepresent example embodiment that enables accurate measurement within ashorter measuring time is significantly useful in accurately performingface authentication with a moving image, for example.

Further, the three-dimensional measuring method according to the presentexample embodiment can perform projection of the short cycle lightpattern and projection of the long cycle light pattern by using a singleprojector device and perform capturing a light pattern projected on ameasured object by using a single image capture device. Therefore, withthe three-dimensional measuring apparatus according to the presentexample embodiment, it is possible to simplify the system configurationor control compared to the conventional method using a plurality ofprojector devices or image capture devices.

Therefore, according to the present example embodiment, it is possibleto realize the three-dimensional shape measuring method and apparatusthat may measure a three-dimensional shape of an object at high accuracyand in a short time with a simpler system.

Second Example Embodiment

A three-dimensional shape measuring apparatus and a three-dimensionalshape measuring method according to a second example embodiment of thepresent invention will be described with reference to FIG. 6A to FIG.9B. The same components as those of the three-dimensional shapemeasuring apparatus according to the first example embodiment will belabeled with the same references, and the description thereof will beomitted or simplified.

FIG. 6A and FIG. 6B are diagrams illustrating examples of a long cyclelight pattern used in the three-dimensional shape measuring methodaccording to the present example embodiment. FIG. 7 is a graphillustrating a calculation method of an absolute phase value in thethree-dimensional shape measuring method according to the presentexample embodiment. FIG. 8 is a flowchart illustrating thethree-dimensional shape measuring method according to the presentexample embodiment. FIG. 9A and FIG. 9B are graphs illustrating resultsof inspection of an advantageous effect of the present invention througha computer simulation.

Although the sinusoidal pattern is used as a short cycle light patternand a long cycle light pattern in the first example embodiment, theshort cycle light pattern and the long cycle light pattern are notnecessarily required to be a sinusoidal pattern and may be anotherperiodic pattern. In particular, various periodic patterns areapplicable to the long cycle light pattern, because it is sufficientthat the position is uniquely defined on the whole screen. In thepresent example embodiment, as an example of the above, a case where asinusoidal pattern is used as the short cycle light pattern and aluminance slope pattern is used as the long cycle light pattern will bedescribed.

Since the derivative near the maximum value and the minimum value of asine wave is close to zero, there is a high likelihood thatsubstantially the same luminance values are projected at adjacentangles. In fact, when 640 pixels, which is a common size of a cameraimage, are projected at 256 levels, which is a common luminancequantized level of a camera or a projector, an error near the maximumvalue or the minimum value of the sine wave increases if the number ofprojecting light patterns is reduced as with the present invention.

To avoid such a problem and remove dependency on a projection angle, itis effective to use a luminance slope pattern whose derivative isconstant, that is, whose luminance value changes linearly at a fixedratio. Accordingly, in the present example embodiment, the luminanceslope pattern is applied as a long cycle light pattern. FIG. 6A and FIG.6B each illustrate an example of the luminance slope pattern. FIG. 6Aillustrates a luminance slope pattern in which the luminance increasesat a fixed ratio from the upper part toward the lower part, and FIG. 6Billustrates a luminance slope pattern in which the luminance decreasesat a fixed ratio from the upper part toward the lower part.

When the measured object 10 on which a luminance slope pattern isprojected is captured by the image capture device 30, the luminancevalue K(x, y, t) at time t at (x, y) coordinates of an obtained image isexpressed as Equation (26) below. Herein, A″ denotes an amplitude, B″denotes a bias, and co is a variable that changes linearly within arange −1≤ω≤1.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 12} \right\rbrack & \; \\{{K\left( {x,y,t} \right)} = {{A^{''} \cdot \left( {{- \left( {{2t} - 1} \right)} \cdot \omega} \right)} + B^{''}}} & (26)\end{matrix}$

Also when the luminance slope pattern is used, Equation (27) below ismet when the same projector device 20 is used for the short cycle lightpattern and the long cycle light pattern. Therefore, if the measuredobject 10 on which the long cycle light pattern is projected is capturedat least once only, the phase value φ can be found.

$\begin{matrix}{{A = A^{''}},{B = B^{''}}} & (27)\end{matrix}$

In the same manner as the case of the first example embodiment, when themeasured object 10 on which the luminance slope pattern is projected iscaptured once only, the luminance value K(x, y, t) and the variable coare expressed by Equation (28) and Equation (29).

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 13} \right\rbrack & \; \\{{K\left( {x,y,0} \right)} = {{A \cdot \omega} + B}} & (28) \\{\omega = \frac{{K\left( {x,y,0} \right)} - B}{A}} & (29)\end{matrix}$

Further, it is possible to perform twice the capturing of the measuredobject 10 on which the luminance slope pattern is projected and find thevariable co by using a least-squares method. That is, if images at timet=0 and time t=1 are obtained, respectively, the luminance value K(x, y,t) at coordinates (x, y) is expressed by Equation (30) and Equation (31)below. The variable co is then expressed by Equation (32) below.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 14} \right\rbrack & \; \\{{K\left( {x,y,0} \right)} = {{A \cdot \omega} + B}} & (30) \\{{K\left( {x,y,1} \right)} = {{{- A} \cdot \omega} + B}} & (31) \\{\omega = \frac{{K\left( {x,y,0} \right)} - {K\left( {x,y,1} \right)}}{2A}} & (32)\end{matrix}$

The variable ω changes gradually in the whole screen and thus isexpected to have a good compatibility with noise removal such assmoothing or a median filter. With design to have the same absolutephase determination accuracy as that when four-time capturing isperformed, it may be possible to cancel the disadvantage caused by areduction of the number of capturing times. Further, unlike the phasevalue, since the variable co is not a periodic value in a strict sense,a classical noise removal process can be utilized without change.

FIG. 7 is a graph illustrating a relationship between the relative phasevalue (phase value θ) and the variable co with respect to the order n ofthe short cycle light pattern when the short cycle light patternincludes 10 cycles and the long cycle light pattern (luminance slopepattern) includes 1 cycle within a screen.

As illustrated in FIG. 7, the order n of the short cycle light patterncan be defined uniquely in accordance with the value of the variable co.For example, the order n of the short cycle light pattern is 2 in arange of the variable co from −0.8 to −0.6, and the order n of the shortcycle light pattern is 7 in a range of the variable co from 0.2 to 0.4.

With the order n of the short cycle light pattern being found, theabsolute phase value Θ (=θ+2π((n−1)) can be calculated from the phasevalue θ and the order n. A line obtained by connecting points having thesame absolute phase value Θ on the captured image (equal-phase line)represents the shape of a cross section of the measured object 10 takenalong a certain plane in a similar manner to a sectional line by alight-section method. Based on this absolute phase value Θ, thethree-dimensional shape of the measured object 10 (height information ateach point of an image) can be measured by a triangulation principle.That is, it is possible to identify the three-dimensional shape of themeasured object 10 by finding absolute coordinate value of a projectionpoint on the measured object in a three-dimensional space correspondingto each pixel on an image by a triangulation principle based on theabsolute phase value Θ and the optical arrangement of the projectordevice 20 and the image capture device 30.

The three-dimensional shape measuring method according to the presentexample embodiment can be performed in accordance with step S201 to stepS216 illustrated in FIG. 8, for example. Note that description isprovided here for a case of calculating the amplitude A, the bias B, andthe phase value θ from an image of the measured object 10 on which thefirst light pattern, which is the short cycle light pattern, isprojected and calculating the phase value φ from an image of themeasured object 10 on which the second light pattern, which is the longcycle light pattern, is projected as an example. However, the amplitudeA′, the bias B′, and the phase value φ may be calculated from an imageof the measured object 10 on which the first light pattern that is thelong cycle light pattern is projected and calculate the phase value θfrom an image of the measured object 10 on which the second lightpattern, which is the short cycle light pattern, is projected.Projection of the second light pattern may be performed earlier thanprojection of the first light pattern.

First, in the same manner as step S101 to step S107 of thethree-dimensional shape measuring method according to the first exampleembodiment, measurement using the first light pattern (short cycle lightpattern) is performed. Thereby, based on the luminance value I of eachpixel of L captured images, the amplitude A, the bias B, and the phasevalue θ are calculated, respectively (step S201 to step S207).

Next, in step S208, the number of projection times M for the secondlight pattern (the luminance slope pattern) is set. Measurement usingthe second light pattern is for determining the variable co, and thenumber of projection times M is greater than or equal to one. To performmore accurate measurement by using a least-squares method or the like,it is desirable that the number of projection times M be greater than orequal to two. As an example here, the number of projection times M forthe second light pattern is two.

Next, in step S209, under the control of the projection pattern controlunit 42, the projector device 20 projects the second light pattern,which is the luminance slope pattern, on the measured object 10. Thepattern of FIG. 6A is applicable as the luminance slope pattern, forexample.

Next, in step S210, under the control of the image acquisition unit 44,the image capture device 30 captures an image of the measured object 10on which the second light pattern is projected.

Next, in step S211, the number of projection times M for the secondlight pattern is decremented by one. The number of projection times Mrepresents the number of remaining projection times of the second lightpattern.

Next, in step S212, it is determined whether or not the number ofprojection times M of the second light pattern is zero, that is, whetheror not the projection and capturing of the second light pattern havebeen performed for the number of projection times M set in step S208.

As a result of the determination of step S212, if the number ofprojection times M is not zero (step S212, “No”), in step S213, thephase of the second pattern projected on the measured object 10 isshifted, and the process returns to step S209. For example, when thenumber of projection times M is two, the projection pattern control unit42 sequentially prepares data of the luminance slope patterns having aninversed luminance slope (see FIG. 6B) and transmits the data to theprojector device 20.

As a result of the determination in step S212, if the number ofprojection times M is zero (step S212, “Yes”), the process proceeds tostep S214.

Next, in step S214, the relative phase value calculation unit 46calculates the variable co based on the amplitude A and the bias Bcalculated in step S207 and the luminance value K of each pixel of the Mimages captured in step S210, respectively. The variable co can becalculated based on Equation (32), for example.

Next, in step S215, the absolute phase value calculation unit 48calculates the absolute phase value Θ based on the phase value θcalculated in step S207 and the variable co calculated in step S214.

Next, in step S216, based on the absolute phase value Θ calculated instep S215, the three-dimensional coordinate calculation unit 50calculates the absolute coordinate values of the projection point on themeasured object 10 in the three-dimensional space corresponding to eachpixel of the captured image. Accordingly, the three-dimensional shape ofthe measured object 10 can be identified.

FIG. 9A and FIG. 9B each illustrate a result of inspection of theadvantageous effect of the present invention through a computersimulation. FIG. 9A illustrates a case of the first example embodimentin which sinusoidal patterns are used as the short cycle light patternand the long cycle light pattern, and FIG. 9B illustrates a case of thepresent example embodiment in which a sinusoidal pattern is used as theshort cycle light pattern and a luminance slope pattern is used as thelong cycle light pattern. In both the cases, it is assumed that thenumber of projection times of the long cycle light pattern is two.

In the computer simulation, after a sinusoidal pattern or a luminanceslope pattern having the amplitude A=64 and the bias B=127 was generatedas an image of 480×640 pixels as a target, normalized noise having amean of 0 and a standard deviation of σ=3 was superimposed thereon,which was quantized within a range of 0 to 255. A sine wave used for theshape measurement was 10-time repetition pattern (see FIG. 2A to FIG.2D), and a sine wave used for the absolute phase determination was ahalf-cycle pattern. In the drawing, the plots×represent the short cyclelight pattern and are illustrated together with an error bar of therange of the standard deviation σ. Further, the plots represent the longcycle light pattern and are illustrated together with an error bar ofthe range of three times the standard deviation σ.

When a sinusoidal pattern is used as a long cycle light pattern, it canbe seen that the accuracy related to determination of the absolute phasevalue significantly drops near the point at which the derivative of thelong cycle light pattern is zero, as illustrated in FIG. 9A. It can beseen that, when a is 3, the six-time range maximum value of the phaseestimation accuracy with a long cycle sinusoidal grating is 33.0% and itwill be difficult to determine the absolute phase unless the cycle ofthe short cycle sinusoidal grating is decreased to around three cycleswithin a screen. As described previously, since the derivative near themaximum value and the minimum value of a sine wave is close to zero,there is a high likelihood that substantially the same luminance valuesare projected at adjacent angles. When the number of projection times ofa light pattern is reduced as with the present invention, the erroraround the maximum value and the minimum value of a sine wave increases.

On the other hand, when a luminance slope pattern is used as a longcycle light pattern, it can be seen that the accuracy related todetermination of the absolute phase value is constant independently ofthe position as illustrated in FIG. 9B and is significantly improved asa whole compared to the case of FIG. 9A. When a is 3, it can be seenthat the six-time range maximum value of the phase estimation accuracywith a long cycle sinusoidal grating is 4.5% and the cycle of the shortcycle sinusoidal grating can be increased up to around 20 cycles withinthe screen. In such a way, by applying a luminance slope pattern as along cycle light pattern, it is possible to reduce a projection angledependency of a measurement error, which enables more accuratemeasurement.

As described above, in the three-dimensional measuring method accordingto the present example embodiment, since the luminance slope pattern isused as the long cycle light pattern, a measurement error can be reducedcompared to the three-dimensional measuring method according to thefirst example embodiment using a sinusoidal pattern as a long cyclepattern. Accordingly, in addition that the advantageous effect providedby the first example embodiment is provided, measurement accuracy can befurther improved.

Therefore, according to the present example embodiment, it is possibleto realize the three-dimensional shape measuring method and apparatusthat may measure the three-dimensional shape of an object at highaccuracy and in a short time with a simpler system.

Third Example Embodiment

A three-dimensional shape measuring apparatus according to a thirdexample embodiment of the present invention will be described withreference to FIG. 10. The same components as those of thethree-dimensional shape measuring apparatus according to the first andsecond example embodiments will be labeled with the same references, andthe description thereof will be omitted or simplified. FIG. 10 is aschematic diagram illustrating a configuration example of thethree-dimensional shape measuring apparatus according to the presentexample embodiment.

The three-dimensional shape measuring apparatus according to the firstand second example embodiments can be configured as illustrated FIG. 10,for example. That is, the three-dimensional shape measuring apparatus100 according to the present example embodiment has a single projectordevice 20 that projects a first light pattern whose luminance changes ata first cycle and a second light pattern whose luminance changes at asecond cycle that is longer than the first cycle on the measured object10. Further, the three-dimensional shape measuring apparatus 100 has theimage capture device 30 that acquires an image of the measured object 10on which the first light pattern or the second light pattern isprojected and the image processing device 40 that processes an imageacquired by the image capture device 30. The image processing device 40has the relative phase value calculation unit 46 that, based on aluminance value at each pixel of an image of the measured object 10 onwhich the first light pattern is projected, calculates a relative phasevalue on each part of the measured object 10 corresponding to eachpixel. Further, the image processing device 40 has the absolute phasevalue calculation unit 48 that, based on the luminance value and therelative phase value at each pixel of an image of the measured object 10on which the second light pattern is projected, calculates an absolutephase value at each part of the measured object 10 corresponding to eachof the pixels. Further, the image processing device 40 has thethree-dimensional coordinate calculation unit 50 that calculatesthree-dimensional coordinates on each part of the measured object 10corresponding to each pixel based on the absolute phase value.

With the above configuration, it is possible to realize thethree-dimensional shape measuring method and apparatus that may measurethe three-dimensional shape of an object at high accuracy and in a shorttime with a simpler system.

Modified Example Embodiments

Various modifications can be made to the present invention without beinglimited to the example embodiments described above.

For example, an example embodiment in which a part of the configurationof any of the example embodiments is added to another example embodimentor an example embodiment in which a part of the configuration of any ofthe example embodiments is replaced with a part of the configuration ofanother example embodiment is one of the example embodiments of thepresent invention.

Further, although the sinusoidal pattern has been illustrated as aperiodic light pattern as an example in the above first and secondexample embodiments, the periodic light pattern is not limited to asinusoidal pattern. For example, a periodic light pattern may be asaw-tooth wave pattern as illustrated in FIG. 11A or may be a triangularwave pattern as illustrated in FIG. 11B. It can be said that theluminance slope pattern illustrated in the second example embodiment isa waveform of one cycle of the saw-tooth wave pattern illustrated inFIG. 11A or a waveform of a half cycle of the triangular wave patternillustrated in FIG. 11B. In view of the above, in the presentspecification, the luminance slope pattern is handled as one of theperiodic light patterns.

Further, although an object that is moving has been illustrated as anexample of the measured object 10 in the above example embodiments, thethree-dimensional shape measuring method and apparatus described in theabove example embodiments can be applied to measurement of the shape ofvarious objects, and the measured object 10 is not limited to a movingobject.

The scope of each of the example embodiments further includes aprocessing method that stores, in a storage medium, a program thatcauses the configuration of each of the example embodiments to operateso as to implement the function of each of the example embodimentsdescribed above, reads the program stored in the storage medium as acode, and executes the program in a computer. That is, the scope of eachof the example embodiments also includes a computer readable storagemedium. Further, each of the example embodiments includes not only thestorage medium in which the program described above is stored but alsothe program itself.

As the storage medium, for example, a floppy (registered trademark)disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, amagnetic tape, a nonvolatile memory card, or a ROM can be used. Further,the scope of each of the example embodiments includes an example thatoperates on OS to perform a process in cooperation with another softwareor a function of an add-in board without being limited to an examplethat performs a process by an individual program stored in the storagemedium.

Note that all the example embodiments described above are mere examplesof embodiment in implementing the present invention, and the technicalscope of the present invention should not be construed in a limitingsense by these example embodiments. That is, the present invention canbe implemented in various forms without departing from the technicalconcept thereof or the primary feature thereof.

The whole or part of the example embodiments disclosed above can bedescribed as, but not limited to, the following supplementary notes.

(Supplementary Note 1)

A three-dimensional shape measuring apparatus comprising:

a single projector device that projects a first light pattern whoseluminance changes at a first cycle and a second light pattern whoseluminance changes at a second cycle that is longer than the first cycleon a measured object;

an image capture device that acquires an image of the measured object onwhich the first light pattern or the second light pattern is projected;and

an image processing device that processes the image acquired by theimage capture device,

wherein the image processing device includes

-   -   a relative phase value calculation unit that, based on a        luminance value at each of pixels of an image of the measured        object on which the first light pattern is projected, calculates        a relative phase value on each part of the measured object        corresponding to each of the pixels,    -   an absolute phase value calculation unit that, based on a        luminance value and the relative phase value at each of pixels        of an image of the measured object on which the second light        pattern is projected, calculates an absolute phase value on each        the part of the measured object corresponding to each of the        pixels, and    -   a three-dimensional coordinate calculation unit that, based on        the absolute phase value, calculates three-dimensional        coordinates at each the part of the measured object        corresponding to each of the pixels.

(Supplementary Note 2)

The three-dimensional shape measuring apparatus according tosupplementary note 1, wherein the image capture device acquires at leastthree images captured with different phases of the first light patternprojected on the measured object and acquires one image in which themeasured object on which the second light pattern is projected iscaptured or two images captured with different phases of the secondlight pattern projected on the measured object.

(Supplementary Note 3)

The three-dimensional shape measuring apparatus according tosupplementary note 1 or 2, wherein the first light pattern is asinusoidal pattern.

(Supplementary Note 4)

The three-dimensional shape measuring apparatus according to any one ofsupplementary notes 1 to 3, wherein the second light pattern is asinusoidal pattern.

(Supplementary Note 5)

The three-dimensional shape measuring apparatus according to any one ofsupplementary notes 1 to 3, wherein the second light pattern is aluminance slope pattern whose luminance changes linearly.

(Supplementary Note 6)

The three-dimensional shape measuring apparatus according tosupplementary note 4 or 5, wherein the second light pattern is a patternby which a whole screen of the image corresponds to one cycle.

(Supplementary Note 7)

The three-dimensional shape measuring apparatus according to any one ofsupplementary notes 1 to 6, wherein the projector device generates thefirst light pattern and the second light pattern from a light emittedfrom a single light source.

(Supplementary Note 8)

The three-dimensional shape measuring apparatus according to any one ofsupplementary notes 1 to 7, wherein the projector device is a DLPprojector or a liquid crystal projector.

(Supplementary Note 9)

The three-dimensional shape measuring apparatus according to any one ofsupplementary notes 1 to 8, wherein the measured object is a movingobject.

(Supplementary Note 10)

The three-dimensional shape measuring apparatus according tosupplementary note 9, wherein the moving object is a face of a person.

(Supplementary Note 11)

A three-dimensional shape measuring method comprising the steps of:

projecting a first light pattern whose luminance changes at a firstcycle on a measured object and acquiring an image of the measured objecton which the first light pattern is projected;

projecting a second light pattern whose luminance changes at a secondcycle that is longer than the first cycle on the measured object by thesame projector device as a projector device used for projection of thefirst light pattern and acquiring an image of the measured object onwhich the second light pattern is projected;

based on a luminance value at each of pixels of an image of the measuredobject on which the first light pattern is projected, calculating arelative phase value on each part of the measured object correspondingto each of the pixels;

based on a luminance value and the relative phase value at each ofpixels of an image of the measured object on which the second lightpattern is projected, calculating an absolute phase value on each thepart of the measured object corresponding to each of the pixels;

and based on the absolute phase value, calculating three-dimensionalcoordinates at each the part of the measured object corresponding toeach of the pixels.

(Supplementary Note 12)

The three-dimensional shape measuring method according to supplementarynote 11,

wherein in the step of acquiring an image of the measured object onwhich the first light pattern is projected, at least three imagescaptured with different phases of the first light pattern projected onthe measured object are acquired, and

wherein in the step of acquiring an image of the measured object onwhich the second light pattern is projected, one image in which themeasured object on which the second light pattern is projected iscaptured or two images captured with different phases of the secondlight pattern projected on the measured object are acquired.

(Supplementary Note 13)

The three-dimensional shape measuring method according to supplementarynote 11 or 12, wherein the first light pattern is a sinusoidal pattern.

(Supplementary Note 14)

The three-dimensional shape measuring method according to any one ofsupplementary notes 11 to 13, wherein the second light pattern is asinusoidal pattern.

(Supplementary Note 15)

The three-dimensional shape measuring method according to any one ofsupplementary notes 11 to 13, wherein the second light pattern is aluminance slope pattern whose luminance changes linearly.

(Supplementary Note 16)

The three-dimensional shape measuring method according to supplementarynote 14 or 15, wherein the second light pattern is a pattern by which awhole screen of the image corresponds to one cycle.

(Supplementary Note 17)

The three-dimensional shape measuring method according to any one ofsupplementary notes 11 to 16, wherein the projector device generates thefirst light pattern and the second light pattern from a light emittedfrom a single light source.

(Supplementary Note 18)

The three-dimensional shape measuring method according to any one ofsupplementary notes 11 to 17, wherein the projector device is a DLPprojector or a liquid crystal projector.

(Supplementary Note 19)

The three-dimensional shape measuring method according to any one ofsupplementary notes 11 to 18, wherein the measured object is a movingobject.

(Supplementary Note 20)

The three-dimensional shape measuring method according to supplementarynote 19, wherein the moving object is a face of a person.

(Supplementary Note 21)

A program that causes a computer to function as:

a unit that controls a single projector device to project a first lightpattern whose luminance changes at a first cycle or a second lightpattern whose luminance changes at a second cycle that is longer thanthe first cycle on a measured object;

a unit that acquires an image of the measured object on which the firstlight pattern is projected and an image of the measured object on whichthe second light pattern is projected;

a unit that, based on a luminance value at each of pixels of an image ofthe measured object on which the first light pattern is projected,calculates a relative phase value on each part of the measured objectcorresponding to each of the pixels;

a unit that, based on a luminance value and the relative phase value ateach of pixels of an image of the measured object on which the secondlight pattern is projected, calculates an absolute phase value on eachthe part of the measured object corresponding to each of the pixels; and

a unit that, based on the absolute phase value, calculatesthree-dimensional coordinates at each the part of the measured objectcorresponding to each of the pixels.

(Supplementary Note 22)

The program according to supplementary note 19, wherein the unit thatacquires the image acquires at least three images captured withdifferent phases of the first light pattern projected on the measuredobject and acquires one image in which the measured object on which thesecond light pattern is projected is captured or two images capturedwith different phases of the second light pattern projected on themeasured object.

(Supplementary Note 23)

A computer readable storage medium storing the program according tosupplementary note 21 or 22.

REFERENCE SIGNS LIST

-   10 . . . measured object-   20 . . . projector device-   30 . . . image capture device-   40 . . . image processing device-   42 . . . projection pattern control unit-   44 . . . image acquisition unit-   46 . . . relative phase calculation unit-   48 . . . absolute phase calculation unit-   50 . . . three-dimensional coordinate calculation unit

1. A three-dimensional shape measuring apparatus comprising: a singleprojector device that projects a first light pattern whose luminancechanges at a first cycle and a second light pattern whose luminancechanges at a second cycle that is different from the first cycle on ameasured object; and an image processing device that processes an imageof the measured object on which the first light pattern or the secondlight pattern is projected, wherein the image processing device includesa relative phase value calculation unit that, based on a luminance valueat each position of an image of the measured object on which the firstlight pattern is projected, calculates a relative phase value on theeach position of the measured object and a parameter of a relationalexpression representing a relationship between a luminance value and aphase value, an absolute phase value calculation unit that, based on aluminance value and the relative phase value at each position of animage of the measured object on which the second light pattern isprojected and the parameter, calculates an absolute phase value on theeach position of the measured object, and a three-dimensional coordinatecalculation unit that, based on the absolute phase value, calculatesthree-dimensional coordinates at the each position of the measuredobject.
 2. The three-dimensional shape measuring apparatus according toclaim 1, wherein the image processing device processes images of a firstnumber captured with different phases of the first light patternprojected on the measured object and processes images of a second numberof less than the first number in which the measured object on which thesecond light pattern is projected is captured.
 3. The three-dimensionalshape measuring apparatus according to claim 1, wherein the imageprocessing device processes at least three images captured withdifferent phases of the first light pattern projected on the measuredobject and processes one image in which the measured object on which thesecond light pattern is projected is captured or two images capturedwith different phases of the second light pattern projected on themeasured object.
 4. The three-dimensional shape measuring apparatusaccording to claim 1, wherein the first light pattern is a sinusoidalpattern.
 5. The three-dimensional shape measuring apparatus according toclaim 1, wherein the second light pattern is a sinusoidal pattern bywhich a whole screen of the image corresponds to one cycle.
 6. Thethree-dimensional shape measuring apparatus according to claim 1,wherein the second light pattern is a luminance slope pattern whoseluminance changes linearly.
 7. The three-dimensional shape measuringapparatus according to claim 1 further includes an image capture devicethat acquires an image of the measured object on which the first lightpattern or the second light pattern is projected.
 8. Thethree-dimensional shape measuring apparatus according to claim 1,wherein the projector device generates the first light pattern and thesecond light pattern from a light emitted from a single light source. 9.The three-dimensional shape measuring apparatus according to claim 1,wherein the projector device is a DLP projector or a liquid crystalprojector.
 10. The three-dimensional shape measuring apparatus accordingto claim 1, wherein the measured object is a moving object.
 11. Thethree-dimensional shape measuring apparatus according to claim 9,wherein the moving object is a face of a person.
 12. A three-dimensionalshape measuring method comprising: projecting a first light patternwhose luminance changes at a first cycle on a measured object andacquiring an image of the measured object on which the first lightpattern is projected; projecting a second light pattern whose luminancechanges at a second cycle that is different from the first cycle on themeasured object by the same projector device as a projector device usedfor projection of the first light pattern and acquiring an image of themeasured object on which the second light pattern is projected; based ona luminance value at each position of an image of the measured object onwhich the first light pattern is projected, calculating a relative phasevalue on the each position of the measured object and a parameter thatis a parameter of a relational expression representing a relationshipbetween a luminance value and a phase value; based on a luminance valueand the relative phase value at each position of an image of themeasured object on which the second light pattern is projected and theparameter, calculating an absolute phase value on the each position ofthe measured object; and based on the absolute phase value, calculatingthree-dimensional coordinates at the each position of the measuredobject.
 13. The three-dimensional shape measuring method according toclaim 11, wherein in the acquiring an image of the measured object onwhich the first light pattern is projected, images of a first numbercaptured with different phases of the first light pattern projected onthe measured object are acquired, and wherein in the acquiring an imageof the measured object on which the second light pattern is projected,images of a second number of less than the first number in which themeasured object on which the second light pattern is projected iscaptured are acquired.
 14. The three-dimensional shape measuring methodaccording to claim 12, wherein in the acquiring an image of the measuredobject on which the first light pattern is projected, at least threeimages captured with different phases of the first light patternprojected on the measured object are acquired, and wherein in theacquiring an image of the measured object on which the second lightpattern is projected, one image in which the measured object on whichthe second light pattern is projected is captured or two images capturedwith different phases of the second light pattern projected on themeasured object are acquired.
 15. The three-dimensional shape measuringmethod according to claim 11, wherein the first light pattern is asinusoidal pattern.
 16. The three-dimensional shape measuring methodaccording to claim 11, wherein the second light pattern is a sinusoidalpattern by which a whole screen of the image corresponds to one cycle.17. The three-dimensional shape measuring method according to claim 11,wherein the second light pattern is a luminance slope pattern whoseluminance changes linearly.
 18. The three-dimensional shape measuringmethod according to claim 12, wherein the projector device is a DLPprojector or a liquid crystal projector.
 19. The three-dimensional shapemeasuring method according to claim 12, wherein the measured object is amoving object.
 20. The three-dimensional shape measuring methodaccording to claim 19, wherein the moving object is a face of a person.